On Learning Decision Heuristics

This document provides details on the random forest implementation and describes the 56 public data sets used in the empirical analysis. Random forest. We used the implementation of random forest in the R package randomForest (Liaw and Wiener, 2002). Typically, the only parameter tuned when using random forests is mtry, which specifies how many attributes should be randomly selected for consideration when splitting a branch (Hastie et al., 2009). We used 10-fold cross-validation to find the best value of mtry from the range 1, 2, ..., k, where k is the number of available attributes. A second important parameter is ntree, which specifies the number of trees built. The default setting in the package (and used frequently in the literature) is 500. We repeated our experiments with ntree set to 500 and to 1,000; we did not see an observable difference. We report results with ntree set to 1,000. Random forests, in addition, have parameters that control how each individual tree is built. In the literature, these parameters are typically not tuned. We used the defaults in the package. In his original paper on random forests, Breiman (2001) recommended setting mtry to b √ kc, where k is the number of attributes. We tried this value as well. With few exceptions, the performance on individual learning curves either declined or remained the same. This result is consistent with the earlier study by Fernández-Delgado et al. (2014). We report only the results obtained when setting mtry using cross-validation. 1. Data Sets AFL Objects: 41 Australian Football League (AFL) games at the Melbourne Cricket Ground in 1993 and 1994. Criterion: attendance. Attributes: forecasted maximum temperature on the day of the game, total attendance at other AFL games in Melbourne and Geelong on the day of the game, total membership in the two clubs whose teams were playing, number of players in the top 50 who participated in the game, number of days since the earliest game of the season. Source: This data set was assembled by Rowan Todd and Mark McNaughton for a class project at the University of Queensland in a statistics course taught by Margaret Mackisack. The data sources were The Football Bible ’94 by Rex Hunt, The Weekend Australian, Inside Football, and Football Record. The data set is